Floppyshop 2

home *** CD-ROM | disk | FTP | other *** search

/ Floppyshop 2 / Floppyshop - 2.zip / Floppyshop - 2.iso / diskmags / 0022-3.564 / dmg-0122 / technica.l / ttns.doc < prev

Wrap

Text File | 1997-04-16 | 16KB | 368 lines

2. TTNS file coding Transmission of character data in the TTNS network has some limitations. Specifically, the TTNS network can only handle 7 bit characters. Furthermore, only a subset of the standard ASCII set can be sued for transparent character transmission. Transmission of 8 bit characters as used in binary source code programs requires a form of encoding for converting each 8 bit character into a sequence of 7 bit characters. Encoding is also required for the transmission of text and source code programs with embedded "control characters". Because of line noise and other factors, files also have to be transmitted with some form of error protection. Sets of characters are organised in blocks which are transmitted as an entity. Each block is appended with a checksum sequence. The receiver checks the validity of every block and flags those which were received in error. A file received with an error count of zero is assumed to have been received correctly. TTNS uses a simple form of encoding, structured as follows: 1. Character encoding: mapping 8 bit bytes into multiple sequences of 7 bit "valid" ASCII haracters. 2. Block encoding: framing of sequences (of already encoded 7 bit character sequences) into blocks surrounded by header and checksum information. 3. File transmission: transmission and acknowledging of sequences of data blocks. The levels are independant and form a subset of each other. Files can be transmitted in character encoded form only or in block form with or without checksums. The format has been developed for line oriented ASCII networks and hosts. Files can be uploaded and downloaded 'blind' from the host, or can be uploaded and downloaded using file transfer programs with acknowledge of blocks and retransmission requests. Files can also be sent as mail items, the integrity of the data will remain intact even if the host (or network) adds extra lines of text, extra messages or reformats the transmitted data (e.g. sections of text such as "--more--' in mail items). Character encoding, decoding The character encoding-decoding procedure allows for transport transmission of 8-bit characters throughout the network. Its main features are:- 1. A very simple decoding algorithm. 2. Transparency and compatability with text files, i.e. encoded text files can be read by equipment without a decoder. 3. One-to-many encoding format, every 8-bit byte can be encoded into more than one character sequence, characters with special meanings can be avoided or transmitted by sending an alternative coded sequence of characters. The encoding procedure is straightforward; bytes which cannot be sent to the line directly, because they have bit 7 set for example, are converted to a sequence of 'valid transmittable' characters. The actual sequence consists of one or more 'escape' characters followed by the original byte 'transformed' by the inversion of one or more of bits 5, 6 or 7 in its pattern. The inversion converts the byte to a character in the 'valid' ASCII range. There are four special escape characters, each of which is used to set up a bit pattern in a flag. The purpose of the flag is to invert (exclusive or) the bit pattern of the next byte. The 'escape' character inverts bits 5, 6 and 7 (in any combination) of the original byte and precedes it. An escape character can follow another, in which case conversion will be performed on the exclusive-or combination ofthe two (or more) escape characters in the sequence. The encoding overhead is, on average, 1.6 to 1 (defined as the ratio of output characters over input bytes), it will be less for files which contain a lot of text information, as text characters are generally in 7-bit byte form. The 'escape' characters themselves are transmitted as dual character sequences. Character Decoding Procedure The decoding procedure is very simple and is the same for all methods of encoding. The program keeps a local variable 'flag' which on initialisation is set to zero, as characters arrive they are tested to see if they belong to the set of escape characters, if they do, bits of the flag are set accordingly. The next received character is exclusive-or'd with the flag, which is then reset to zero. In pseudo-program form a procedure which takes as input character encoded data, and saves to disc the recovered 8-bit bytes... DECODE (input CHAR, local FLAG (initialise to zero)) if (CHAR=x'7B') then FLAG = FLAG xor x'80'; if (CHAR=x'7C') then FLAG = FLAG xor x'40'; if (CHAR=x'7D') then FLAG = FLAG xor x'A0'; if (CHAR not in [x'7B' .. x'7E']) then CHAR = CHAR xor FLAG write_to_file (CHAR) FLAG = 0; end; In other words, all received characters are passed straight through until one of the four special escape characters is received. The received escape character modifies an internal flag and is not saved to disc. The next received non-escape character is exclusive-or'd with the internal flag and saved to disc; the flag being cleared afterwards. More than one escape character can be received in a sequence, each one modifying the flag accordingly until a non-escape character arrives. Character Encoding Procedure Bytes can be encoded in more than one way (on decoding all methods will produce the same results). The only basic rule is that the encoding process must not allow any 'illegal' characters to be sent. The basic encoding procedure is as follows:- If the byte is in the 'valid range' then transmit without modification, otherwise transmit an escape character followed by the byte modified (by exclusive-or'ing one of the bits), so that it is in the valid range. The encoding table (including alternatives) is as follows:- Input byte Standard encoded output + some alternatives range 00 - 1F <7E><20-3F> or <7C><40-5F> or <7C><7E><60-7B> 20 - 3F <20-3F> or <7C><60-7B> or <7C><7E><40-5F> 40 - 5F <40-5F> or <7E><60-7B> or <7C><7E><20-3F> 60 - 7A <60-7A> or <7C><20-3F> or <7E><40-5F> 7B - 7F <7C><3B-3F> or <7E><5B-5F> 80 - 9F <7D><20-3F> or <7B><7C><40-5F> A0 - BF <7B><20-3F> or <7C><7D><40-5F> C0 - DF <7B><40-5F> or <7D><60-7B> or <7C><7D><20-3F> E0 - FF <7D><40-5F> or <7B><60-7B> or <7B><7C><20-3F> The program to read an 8-bit byte from disc and return converted characters is as follows. Note...The program requires a local storage flag plus a one character storage location. Initialisation requires the flag to be set to zero. ENCODE (output CHAR; local FLAG, CHOLD (initialise to zero)) if (FLAG <> 0) then CHAR = CHOLD xor FLAG; FLAG = 0; else read-byte-from-file into CHOLD if (CHOLD in [x'00' .. x'1F']) then FLAG = x'40'; CHAR = x'7C'; if (CHOLD in [x'20' .. x'7A']) then FLAG = x'00'; CHAR = CHOLD; if (CHOLD in [x'7B' .. x'7F']) then FLAG = x'40'; CHAR = x'7C'; if (CHOLD in [x'80' .. x'9F']) then FLAG = x'A0'; CHAR = x'7D'; if (CHOLD in [x'A0' .. x'DF']) then FLAG = x'80'; CHAR = x'7B'; if (CHOLD in [x'E0' .. x'FF']) then FLAG = x'A0'; CHAR = x'7D'; end; It is possible to encode an 8-bit byte into more than one different combination of output characters. The encoder has therefore more than one encoding choice for every particular character, this is used to avoid the transmission of 'unwelcome' characters, such as 'commercial at', which is used as a line delete command in Dialcom software. For example, the byte <00> would normally be encoded as <7C><40> which includes the '@' sign; the program above would be modified to encode <00> to <7E><20> instead. Block Encoding Decoding For the downloading of files, character encoded data is divided into variable length records and is then 'bracketed' by block separators. The purpose of the blocks is to divide the data into lengths shorter than the defined line length of the host or network. Blocks also allowfor the interleaving of data which is not part of the file but may be required by other parts of the system or network, including carriage return, line feed and other formatting characters. Blocks are formatted as follows:- <sq><7B><7B> ...encoded data... <7D><7D><ck><ck> Data outside the blocks is not considered part of the file and is ignored (this data will usually contain carriage returns etc. for formatting purposes). The block is preceded by an optional sequence character. Two checksum characters can optionally follow the block. Simple decoders can ignore the sequence character and the checksums (they are outside the block). The (optional) sequence character is a single digit (0..7), x'30' to x'37'. A block is transmitted with character '0', the next with '1' etc. After 7, the next block continues with '0' and so on. The purpose of sequence characters is to ensure that blocks are received in the correct order. Simple decoders can ignore the sequence character (aletrnatively, they could perform a simple test on the least significant bit). If the receiver does not recognise a consistant ordered sequence, it assumes that the blocks were not preceded by sequence characters and will ignore the check. Again this is to allow for simple encoder-decoder operation. The checksum pair is two hex digits <00..FF> corresponding to the xor total of all the bytes between the brackets. The decoder has the option of calculating checksums for every block (and reporting errors to the user) or ignoring the checksums altogether. If the checksum is not appended to the block, the receiver will recognise the fact and perform only a simple parity check; simple transmitters can send the checksum <00> for all blocks. This allows for full flexibility in checksum protection. The checksum is easy to calculate by the decoder. Error situations expected on the network can be one of the following: single or multiple error bits in a block (random noise), burst errors causing whole sections of blocks not to be received, delay conditions where blocks arrive late, or do not arrive at all, and complete loss of transmission. Overall error protection in this scheme is provided by the fact that all characters are transmitted in 7-bit even parity form. The decoder provides a check on both horizontal and vertical checksums. Loss of block sequencing is also detected by means of the sequencing digit. This overall error protecting scheme provides a substantially reliable form for data protection, even for simple decoders ignoring the horizontal checksum. When the whole file has been received, the decoder closes the local disc file and notifies the user of the total number of errors received. For 'blind' transmission, the user will then request for the whole file to be transmitted again from the host. Data Header Block The header block is as follows: <sq><7C><7C>..header information..<7D><7D><ck><ck> Note, <sq> can be character '7' (x'37') to force the first data block to have a sequence character of zero. Simple decoders can ignore the header block altogether. The end of file block is as follows: <sq><7B><7B><7E><7E> The disc file is automatically closed. For example ... 7||FREDF.COM aug 13 L1800,R1200}}31 0{{this is a line of text|M|J this is the next line|M|J}}43 1{{and another line|M|J}}12 2{{this is the last line|M|J}}91 3{{~~}} A simple block decode procedure is as follows: 1. ignore all received characters until '{{' or '||' is received. 2. reset all flags etc. 3. if data pass all received data to character decoder procedure. 4. if header, set local header information. 5. if '}}' is received, read checksum (optional) and goto 1 above. 6. if '~~' is received, close the file and end. A simplified block encode procedure is as follows: 1. set n=7. 2. Transmit header block i.e. 'n||MBBC}}12'. 3. increment n mod 7. 4. transmit 'n{{'. 5. transmit character encoded data until say 80 bytes or EOL. 6. transmit '}}' plus two or four checksum characters (if required). 7. transmit interblock data as required (comments, CR-LF's etc. 8. repeat from 3 above. 9. at end of file transmit 'n{{~~'. Header information: A typical header.. 7||MBBC,FGAME.BAS,L1E00,R8023}}C3 The fields start with a single character and are separated by commas (all fields are optional). The fields are: Mnnn Machine type (i.e. MBBC, MRML, MACT, MIBM etc) Fnnnn Filename Lnnnn Load address in hex (any length) Rnnnn Run address in hex (any length) Cxxxxx Comment line 3. TTNS format control codes This option allows characters embedded in the source text to control facilities at the terminal end such as printer control, file control and screen pauses. This facility has a number of applications: - Automatic software downloading .. avoids the user having to type out the complex sequences presently requireed. - Automatic control of printer. Downloaded information can be selective in the way it presents information for display and for remote printing. - Screen pauses allow the user to control the stop point of his document, the users can continue input by pressing any key. That appears to be that!